Search results for "Storage management"
showing 4 items of 4 documents
Random Slicing: Efficient and Scalable Data Placement for Large-Scale Storage Systems
2014
The ever-growing amount of data requires highly scalable storage solutions. The most flexible approach is to use storage pools that can be expanded and scaled down by adding or removing storage devices. To make this approach usable, it is necessary to provide a solution to locate data items in such a dynamic environment. This article presents and evaluates the Random Slicing strategy, which incorporates lessons learned from table-based, rule-based, and pseudo-randomized hashing strategies and is able to provide a simple and efficient strategy that scales up to handle exascale data. Random Slicing keeps a small table with information about previous storage system insert and remove operations…
LPCC
2019
Most high-performance computing (HPC) clusters use a global parallel file system to enable high data throughput. The parallel file system is typically centralized and its storage media are physically separated from the compute cluster. Compute nodes as clients of the parallel file system are often additionally equipped with SSDs. The node internal storage media are rarely well-integrated into the I/O and compute workflows. How to make full and flexible use of these storage media is therefore a valuable research question. In this paper, we propose a hierarchical Persistent Client Caching (LPCC) mechanism for the Lustre file system. LPCC provides two modes: RW-PCC builds a read-write cache on…
NVMM-Oriented Hierarchical Persistent Client Caching for Lustre
2021
In high-performance computing (HPC), data and metadata are stored on special server nodes and client applications access the servers’ data and metadata through a network, which induces network latencies and resource contention. These server nodes are typically equipped with (slow) magnetic disks, while the client nodes store temporary data on fast SSDs or even on non-volatile main memory (NVMM). Therefore, the full potential of parallel file systems can only be reached if fast client side storage devices are included into the overall storage architecture. In this article, we propose an NVMM-based hierarchical persistent client cache for the Lustre file system (NVMM-LPCC for short). NVMM-LPC…
Fusing storage and computing for the domain of business intelligence and analytics: research opportunities
2015
With the growing importance of external and shared data, the set of requirements for Business Intelligence and Analytics (BIA) is shifting. Current solutions still come with shortcomings, esp. In multi-stakeholder environments where sensitive content is exchanged. We argue that a new level in the evolution of BIA can be unlocked by tearing down the barriers between storage and computing based on upcoming storage technologies. In particular, we propose a revitalization of ideas from object-oriented databases. We present results from a joint project that aimed at delineating design options for BIA solutions built upon this idea. The paper outlines the interplay of various architectural layers…